airflow.providers.cncf.kubernetes.secrets.kubernetes_secrets_backend

Objects relating to sourcing connections, variables, and configs from Kubernetes Secrets.

Classes

KubernetesSecretsBackend

Retrieve connections, variables, and configs from Kubernetes Secrets using labels.

Module Contents

class airflow.providers.cncf.kubernetes.secrets.kubernetes_secrets_backend.KubernetesSecretsBackend(namespace=None, connections_label=DEFAULT_CONNECTIONS_LABEL, variables_label=DEFAULT_VARIABLES_LABEL, config_label=DEFAULT_CONFIG_LABEL, connections_data_key='value', variables_data_key='value', config_data_key='value', **kwargs)[source]

Bases: airflow.secrets.BaseSecretsBackend, airflow.utils.log.logging_mixin.LoggingMixin

Retrieve connections, variables, and configs from Kubernetes Secrets using labels.

This backend discovers secrets by querying Kubernetes labels, enabling integration with External Secrets Operator (ESO), Sealed Secrets, or any tool that creates Kubernetes secrets — regardless of the secret’s name.

Configurable via airflow.cfg:

[secrets]
backend = airflow.providers.cncf.kubernetes.secrets.kubernetes_secrets_backend.KubernetesSecretsBackend
backend_kwargs = {"namespace": "airflow", "connections_label": "airflow.apache.org/connection-id"}

The secret must have a label whose key matches the configured label and whose value matches the requested identifier (conn_id, variable key, or config key). The actual secret value is read from the value key in the secret’s data.

Example Kubernetes secret for a connection named my_db:

apiVersion: v1
kind: Secret
metadata:
  name: anything
  labels:
    airflow.apache.org/connection-id: my_db
data:
  value: <base64-encoded-connection-uri>

Authentication: Uses kubernetes.config.load_incluster_config() directly for in-cluster authentication. Does not use KubernetesHook or any Airflow connection, avoiding circular dependencies since this IS the secrets backend. The namespace can be set explicitly via backend_kwargs. If not set, it is auto-detected from the pod’s service account metadata at /var/run/secrets/kubernetes.io/serviceaccount/namespace. If auto-detection fails (e.g. automountServiceAccountToken is disabled), an error is raised.

Performance: Queries use resource_version="0" so the Kubernetes API server serves results from its in-memory watch cache, making lookups very fast without requiring Airflow-side caching.

Parameters:

  • namespace (str | None) – Kubernetes namespace to query for secrets. If not set, the namespace is auto-detected from the pod’s service account metadata. If auto-detection fails, an AirflowException is raised.

  • connections_label (str) – Label key used to discover connection secrets. If set to None, requests for connections will not be sent to Kubernetes.

  • variables_label (str) – Label key used to discover variable secrets. If set to None, requests for variables will not be sent to Kubernetes.

  • config_label (str) – Label key used to discover config secrets. If set to None, requests for configurations will not be sent to Kubernetes.

  • connections_data_key (str) – The data key in the Kubernetes secret that holds the connection value. Default: "value"

  • variables_data_key (str) – The data key in the Kubernetes secret that holds the variable value. Default: "value"

  • config_data_key (str) – The data key in the Kubernetes secret that holds the config value. Default: "value"

DEFAULT_CONNECTIONS_LABEL = 'airflow.apache.org/connection-id'[source]
DEFAULT_VARIABLES_LABEL = 'airflow.apache.org/variable-key'[source]
DEFAULT_CONFIG_LABEL = 'airflow.apache.org/config-key'[source]
SERVICE_ACCOUNT_NAMESPACE_PATH = '/var/run/secrets/kubernetes.io/serviceaccount/namespace'[source]
connections_label = 'airflow.apache.org/connection-id'[source]
variables_label = 'airflow.apache.org/variable-key'[source]
config_label = 'airflow.apache.org/config-key'[source]
connections_data_key = 'value'[source]
variables_data_key = 'value'[source]
config_data_key = 'value'[source]
property namespace: str[source]

Return the configured namespace, or auto-detect from service account metadata.

property client: kubernetes.client.CoreV1Api[source]

Lazy-init Kubernetes CoreV1Api client using in-cluster config directly.

get_conn_value(conn_id, team_name=None)[source]

Get serialized representation of Connection from a Kubernetes secret.

Multi-team isolation is not currently supported; team_name is accepted for API compatibility but ignored.

Parameters:

  • conn_id (str) – connection id

  • team_name (str | None) – Team name (unused — multi-team is not currently supported)

get_variable(key, team_name=None)[source]

Get Airflow Variable from a Kubernetes secret.

Multi-team isolation is not currently supported; team_name is accepted for API compatibility but ignored.

Parameters:

  • key (str) – Variable Key

  • team_name (str | None) – Team name (unused — multi-team is not currently supported)

Returns:

Variable Value

Return type:

str | None

get_config(key)[source]

Get Airflow Configuration from a Kubernetes secret.

Parameters:

key (str) – Configuration Option Key

Returns:

Configuration Option Value

Return type:

str | None

Was this entry helpful?