Airflow Summit 2025 is coming October 07-09. Register now for early bird ticket!

Supported classes

Below is a list of Operators and Hooks that support OpenLineage extraction, along with specific DB types that are compatible with the SQLExecuteQueryOperator.

Important

While we strive to keep the list of supported classes current, please be aware that our updating process is automated and may not always capture everything accurately. Detecting hook level lineage is challenging so make sure to double check the information provided below.

Tip

You can easily implement OpenLineage support for any operator. See Implementing OpenLineage in Operators.

Core operators

At the moment, two core operators support OpenLineage. These operators function as a ‘black box,’ capable of running any code, which might limit the extent of lineage extraction. To enhance the extraction of lineage information, operators can utilize the hooks listed below that support OpenLineage.

Spark operators

The OpenLineage integration can automatically inject information into Spark application properties when its being submitted from Airflow. The following is a list of supported operators along with the corresponding information that can be injected. See automatic injection of parent job information for more details.

apache-airflow-providers-apache-livy

  • LivyOperator
    • Parent Job Information

    • Transport Information (only HTTP transport is supported for now (with api_key auth, if any))

apache-airflow-providers-apache-spark

  • SparkSubmitOperator
    • Parent Job Information

    • Transport Information (only HTTP transport is supported for now (with api_key auth, if any))

apache-airflow-providers-google

SQLExecuteQueryOperator and derivatives

These operators are using SQL parsing and may query DB for lineage extraction. To extract unique data from each database type, a dedicated Hook implementing OpenLineage methods is required.

Note

Not all subclasses of SQLExecuteQueryOperator are automatically supported, only those also using a supported Hook (e.g., one of supported operators is DatabricksSqlOperator). Due to the automatic generation of this documentation, some supported operators inheriting from SQLExecuteQueryOperator may not appear in the Providers section below - to prevent false positives. Please check your operator’s code to confirm OpenLineage support.

Currently, the following databases (hooks) are supported:

Providers

The operators and hooks listed below from each provider are natively equipped with OpenLineage support.

apache-airflow-providers-amazon (9.8.0)

Operators

Hooks

apache-airflow-providers-common-io (1.6.0)

Operators

apache-airflow-providers-databricks (7.4.0)

Operators

apache-airflow-providers-dbt-cloud (4.4.0)

Operators

apache-airflow-providers-ftp (3.13.0)

Operators

apache-airflow-providers-google (15.1.0)

Operators

Hooks

apache-airflow-providers-sftp (5.3.0)

Operators

apache-airflow-providers-snowflake (6.3.1)

Operators

Was this entry helpful?