What's new in Apache Airflow 2.8?

Опубликовано: 19 Октябрь 2024
на канале: Data with Marc
3,076
103

What's new in Apache Airflow 2.8?

Join me on Jan 11th, 2024 to discover Airflow 2.8 new features in PRACTICE 👇
https://www.astronomer.io/events/webi...

Here are the significant changes:

➡️ New XComs tab!
💻 Access data shared by tasks directly from the Grid view

➡️ Task context logging
💻 A task in failure? Access messages from Airflow components (Scheduler, Executor) to the task logs to know what happened all at a single place

➡️ New listeners to datasets
💻 Take actions on dataset creations and updates

➡️ New Airflow ObjectStore API
💻 Seamlessly transfer data between different object storage systems with a unified interface. No more transfer operators

➡️ New prev_start_date_success
💻 Take actions based on the data interval of the previous successful DAG run

➡️ New BranchPythonVirtualEnvOperator
💻 Choose between different tasks without dependency conflicts

➡️ New Decorators
💻 For external and virtual env branch operators

➡️ New clear_number attribute
💻 Track the number of times a DAG run has been cleared (restarted)

➡️ Allow override of navbar text color
💻 Use NAVBAR_TEXT_COLOR to change the navbar text color and identify environments

➡️ Description of pools!
💻 Never forget what a pool does with the description field

And more!

As usual, BIG shoutout to all contributors and the vibrant community!

We’re getting closer to 3.0 🥹

Join me on Jan 11th, 2024 to discover Airflow 2.8 new features in PRACTICE 👇
https://www.astronomer.io/events/webi...

🏆 BECOME A PRO: https://www.udemy.com/course/the-comp...

👍 Smash the like button to become an Airflow Super Hero!
❤️ Subscribe to my channel to become a master of Airflow

🚨 My Patreon:   / marclamberti  

Ready?

Let's go!