From the course: Apache Airflow Essential Training
Unlock this course with a free trial
Join today to access over 24,700 courses taught by industry experts.
Branching using the TaskFlow API - Apache Airflow Tutorial
From the course: Apache Airflow Essential Training
Branching using the TaskFlow API
- [Instructor] I have a new DAG here in a new file, branching_using_taskflow. This is where we'll see how we can perform branching using the Taskflow API. We already know how to do this with operators. Now, in this first iteration of this demo, I'm only going to convert the branch operator to use the Taskflow API. We'll work with traditional Python operators for the rest of our Python code. This will also allow you to see how you can have the Taskflow API interoperate with traditional operators. Let's scroll down and take a look at the code. Here is my function definition, branching_using_taskflow on line 23. I've added the @dag decorator to this function, because I'm using the Taskflow API here. And of course, the @dag decorator specifies the dag ID description and a number of other details. My read CSV file function reads the card data CSV as before. This function is just a Python callable using Python operators…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
Branching using operators6m 18s
-
(Locked)
Branching using the TaskFlow API4m 45s
-
(Locked)
Complete branching pipeline using TaskFlow3m 17s
-
(Locked)
Interoperating and passing data between operators and TaskFlow4m 53s
-
(Locked)
Performing SQL operations with TaskFlow4m 53s
-
(Locked)
Data transformation and storage using TaskFlow4m 23s
-
-
-
-
-