Uploaded by Suhail Imran

4. Understanding Airflow User Interface by Najma Bader Medium

advertisement
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Member-only story
4. Understanding Airflow User Interface
My personal notes from the book “Data Pipelines with Apache Airflow” by Bas
Harenslak and Julian de Ruiter — Chapter 2, Part 4
Najma Bader · Follow
5 min read · Oct 24, 2022
Listen
Share
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
1/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Data Pipelines with Apache Airflow — Manning Publications
This series of posts is meant to summarize my learnings from the book by Bas
Harenslak and Julian de Ruiter. If you like the content, you can purchase the book
on Manning.
Complete list:
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
2/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Chapter 2:
- 1. Introduction to Airflow
- 2. Running Airflow Locally (in a Python Environment)
- 3. Running Airflow with Docker
- 4. Understanding Airflow User Interface
Airflow UI
Whether you are running Airflow in a Python environment or using Docker, it’s
standard practice to forward Airflow on port 8080 of the host machine (i.e. your
computer).
Once you have Airflow up and running, open a browser and go to
http://localhost:8080
. The first screen you will see is a login screen. Enter your
Username and Password. If you followed the instructions of the previous posts, they
should be
Username=admin
and
Password=admin
.
Airflow login
Once you enter, you might see a lot of DAGs. Don’t panic. These are examples
automatically loaded.
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
3/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Open in app
Sign up
Sign In
Airflow Home
Let’s just focus on the DAG we have created
download_rocket_launches
. If you click
on the DAG name you should see a screen like this:
On the top of the page, you can see many different visualisations.
The most commonly used are the Code View, the Graph View, and the Tree View.
Code View
Read the full story with a free account.
The author made this story available to Medium members only.
Sign up to read this one for free.
Continue in app
Or, continue in mobile web
Sign up with Google
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
4/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Sign up with Facebook
Sign up with email
Already have an account? Sign in
More from the list: "Spatial Data Engineering"
Curated by Sutan Mufti
Adarsh D… in Mobile App…
Akib Hobale in Globant
The Educat… in Dev Le
Securing Node JS: Best
Practices and Tips
Node.js Buffer vs. Stream
Multithreading and
concurrency…
4 min read · Mar 17
6 min read · May 2
8 min read · Apr 7
View list
Follow
Written by Najma Bader
329 Followers
I grew up in a hippie family but I fell in love with coding. I write about my experience with Python, Big Data
and other nice things ❤️
More from Najma Bader
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
5/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Najma Bader in Towards Data Science
The Good way to structure a Python Project
Some Python advices I wish I knew as a data beginner
· 6 min read · Feb 10, 2022
599
4
Najma Bader
How to create a Linux Virtual Machine on macOS
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
6/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Table of Contents:
· 6 min read · Jan 17
4
Najma Bader
How to define the DAG “schedule_interval” parameter
My personal notes from the book “Data Pipelines with Apache Airflow” by Bas Harenslak and
Julian de Ruiter — Chapter 3, Part 2
· 5 min read · Oct 27, 2022
8
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
7/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Najma Bader
12. Writing an Airflow PythonOperator with Jinja templates
My personal notes from the book “Data Pipelines with Apache Airflow” by Bas Harenslak and
Julian de Ruiter — Chapter 4, Part 1
· 4 min read · Oct 31, 2022
9
See all from Najma Bader
Recommended from Medium
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
8/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Agusmahari
Apache Airflow : 20 examples of tasks that are often implemented using
the BashOperator in Apache…
Wondering how can you execute bash commands through Airflow
2 min read · Mar 26
18
2
Jeff Fletcher in Apache Airflow
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
9/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Passing Data Between Tasks with the KubernetesPodOperator in Apache
Airflow
TL;DR: Use the @task.kubernetes decorator!
8 min read · Mar 21
60
1
Lists
New_Reading_List
174 stories · 96 saves
It's never too late or early to start something
15 stories · 107 saves
General Coding Knowledge
20 stories · 306 saves
Predictive Modeling w/ Python
20 stories · 361 saves
Marin Aglić in Better Programming
Making Async API Calls With Airflow Dynamic Task Mapping
11 min read · Apr 18
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
10/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
2
Rajesh Ku. Rout in AWS Tip
Integrating Slack with Airflow: Step-by-Step Guide to Setup Alerts
As data pipelines become more complex, it’s important to monitor them closely to ensure they
are running smoothly. One way to do this is to…
7 min read · Apr 5
70
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
11/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Abdullah Siddique in Dev Genius
Intro To Airflow: Setting Up Apache Airflow
Introduction
5 min read · May 4
15
Maroof Ashraf
Installing Apache Airflow on Windows: A Step-by-Step Guide
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
12/13
9/10/23, 1:29 AM
4. Understanding Airflow User Interface | by Najma Bader | Medium
Introduction
3 min read · Jul 19
5
See more recommendations
https://medium.com/@thehippieandtheboss/4-understanding-airflow-user-interface-e2320ec1bf3e
13/13
Download