_images/helm-logo.svg

Helm Chart for Apache Airflow

This chart will bootstrap an Airflow deployment on a Kubernetes cluster using the Helm package manager.

Requirements

  • Kubernetes 1.20+ cluster

  • Helm 3.0+

  • PV provisioner support in the underlying infrastructure (optionally)

Features

  • Supported executors: LocalExecutor, CeleryExecutor, CeleryKubernetesExecutor, KubernetesExecutor.

  • Supported Airflow version: 1.10+, 2.0+

  • Supported database backend: PostgresSQL, MySQL

  • Autoscaling for CeleryExecutor provided by KEDA

  • PostgreSQL and PgBouncer with a battle-tested configuration

  • Monitoring:

    • StatsD/Prometheus metrics for Airflow

    • Prometheus metrics for PgBouncer

    • Flower

  • Automatic database migration after a new deployment

  • Administrator account creation during deployment

  • Kerberos secure configuration

  • One-command deployment for any type of executor. You don’t need to provide other services e.g. Redis/Database to test the Airflow.

Installing the Chart

To install this chart using Helm 3, run the following commands:

kubectl create namespace airflow
helm repo add apache-airflow https://airflow.apache.org
helm install airflow apache-airflow/airflow --namespace airflow

The command deploys Airflow on the Kubernetes cluster in the default configuration. The Parameters reference section lists the parameters that can be configured during installation.

Tip

List all releases using helm list.

Upgrading the Chart

To upgrade the chart with the release name airflow:

helm upgrade airflow apache-airflow/airflow --namespace airflow

Note

To upgrade to a new version of the chart, run helm repo update first.

Uninstalling the Chart

To uninstall/delete the airflow deployment:

helm delete airflow --namespace airflow

The command removes all the Kubernetes components associated with the chart and deletes the release.

Installing the Chart with ArgoCD

When installing the chart using ArgoCD, you MUST set the two following values, or your application will not start as the migrations will not be run:

createUserJob.useHelmHooks: false
migrateDatabaseJob.useHelmHooks: false

Was this entry helpful?