Building Airflow CTL with Buğra Öztürk at Mollie
Buğra Öztürk, Senior Data Engineer atMollie and Committer and PMC member on the Apache Airflow project, joins us to walk through Airflow CTL — what it is, how it differs from the existing Airflow CLI and where it is headed under AIP-94.
Key Takeaways:
00:00 Introduction.
03:10 Buğra has contributed to Airflow since 2022, from docs changes up to Committer and PMC member — a path he hopes inspires others to start small and contribute.
04:05 Airflow CTL solves secure user interaction by abstracting database credentials behind the public core API.
05:13 Airflow CLI and Airflow CTL are complementary — CLI handles administration and database management while CTL handles secure user interactions via the API.
07:08 Airflow CTL authenticates via the API, acquires a JWT token and stores it securely in the OS keyring — running on the user's machine and never requiring direct database access.
08:21 Concrete use cases include local DAG development without the UI and CI/CD automation using headless mode with short-lived JWT tokens.
10:08 AIP-94 describes the long-term vision — decoupling all remote commands from the Airflow CLI and routing them through Airflow CTL.
13:12 Airflow CTL is currently at 0.X and already being used in CI and deployment automations. The move to 1.0 with full CLI parity is the next milestone under AIP-94.
16:09 Multi-team deployment becoming generally available in a future Airflow release is Buğra's most-anticipated upcoming feature beyond Airflow CTL.
Resources Mentioned:
https://www.linkedin.com/in/bugraozturk93/
https://www.linkedin.com/company/mollie/
Mollie | Website
https://www.mollie.com/
https://airflow.apache.org/
https://lists.apache.org/thread/d2o1pr78wxdp1wozq519stp0pkcv6k6c
https://www.github.com/apache/airflow
Thanks for listening to “The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
#AI #Automation #Airflow #MachineLearning
Fler avsnitt av The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI
Visa alla avsnitt av The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AIThe Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI med Astronomer finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.