# Airflow Survey 2019

DevFeed: [Airflow Survey 2019](<https://devfeed.tech/articles/airflow-survey-2019-32549.md>)

Original publisher: [Read original article](<https://airflow.apache.org/blog/airflow-survey/>)

Author: Apache Airflow

Published: 2019-12-11T00:00:00Z

Content type: article

Language: en

Sources: [Apache Airflow Blog](<https://devfeed.tech/sources/apache-airflow-blog.md>)

Topics: [airflow](<https://devfeed.tech/topics/airflow.md>), [data-processing](<https://devfeed.tech/topics/data-processing.md>), [etl](<https://devfeed.tech/topics/etl.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [DevOps](<https://devfeed.tech/topics/devops.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>), [ui](<https://devfeed.tech/topics/ui.md>)

Tags: [apache-airflow](<https://devfeed.tech/tags/apache-airflow.md>), [cli](<https://devfeed.tech/tags/cli.md>), [community](<https://devfeed.tech/tags/community.md>), [data](<https://devfeed.tech/tags/data.md>), [devops](<https://devfeed.tech/tags/devops.md>), [etl](<https://devfeed.tech/tags/etl.md>), [overview](<https://devfeed.tech/tags/overview.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [survey](<https://devfeed.tech/tags/survey.md>), [testing](<https://devfeed.tech/tags/testing.md>), [ui](<https://devfeed.tech/tags/ui.md>), [users](<https://devfeed.tech/tags/users.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

## AI overview

This article reports findings from the 2019 Apache Airflow user survey, which received 308 responses. Respondents primarily used Airflow for data processing and ETL, with additional use for machine learning pipelines and DevOps operations. The survey also describes the scale of users' DAGs and tasks, onboarding difficulties, testing and UI concerns, and interface usage.

## Source excerpt

Apache Airflow Survey 2019 Apache Airflow is growing faster than ever. Thus, receiving and adjusting to our users' feedback is a must. We created survey and we got 308 responses. Let's see who Airflow users are, how they play with it, and what they miss. Overview of the user What best describes your current occupation? No. % Data Engineer 194 62.99% Developer 34 11.04% Architect 23 7.47% Data Scientist 19 6.17% Data Analyst 13 4.22% DevOps 13 4.22% IT Administrator 2 0.65% Machine Learning Engineer 2 0.65% Manager 2 0.65% Operations 2 0.65% Chief Data Officer 1 0.32% Engineering Manager 1 0.32% Intern 1 0.32% Product owner 1 0.32% Quant 1 0.32% In your day to day job, what do you use Airflow for? No. % Data processing (ETL) 298 96.75% Artificial Intelligence and Machine Learning Pipelines 90 29.22% Automating DevOps operations 64 20.78% According to the survey, most of the Airflow users are the "data" people. Moreover, 28.57% uses Airflow to both ETL and ML pipelines meaning that those two fields are somehow connected. Only five respondents use Airflow for DevOps operations only, That means that other 59 people who use Airflow for DevOps stuff use it also for ETL / ML purposes. How many active DAGs do you have in your largest Airflow instance? No. % 0-20 115 37.34% 21-40 65 21.10% 41-60 44 14.29% 61-100 28 9.09% 101-200 28 9.09% 201-300 7 2.27% 301-999 8 2.60% 1000+ 13 4.22% The majority of users do not exceed 100 active DAGs per Airflow instance. However, as we can see there are users who exceed thousands of DAGs with a maximum number 5000. What is the maximum number of tasks that you have used in one DAG? No. % 0-10 61 19.81% 11-20 60 19.48% 21-30 31 10.06% 31-40 21 6.82% 41-50 26 8.44% 51-100 36 11.69% 101-200 28 9.09% 201-500 21 6.82% 501+ 24 11.54% The given maximum number of tasks in a single DAG was 10 000 (!). The number of tasks depends on the purposes of a DAG, so it's rather hard to say if users have "simple" or "complicated" workflows. When onboarding ne