airflow.providers.openlineage.plugins.adapter

Module Contents

Classes

OpenLineageAdapter

Translate Airflow metadata to OpenLineage events instead of creating them from Airflow code.
class airflow.providers.openlineage.plugins.adapter.OpenLineageAdapter(client=None, secrets_masker=None)[source]

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Translate Airflow metadata to OpenLineage events instead of creating them from Airflow code.

get_or_create_openlineage_client()[source]
get_openlineage_config()[source]
static build_dag_run_id(dag_id, logical_date, clear_number)[source]
static build_task_instance_run_id(dag_id, task_id, try_number, logical_date, map_index)[source]
emit(event)[source]

Emit OpenLineage event.

Parameters

event (openlineage.client.event_v2.RunEvent) – Event to be emitted.

Returns

Redacted Event.

start_task(run_id, job_name, job_description, event_time, parent_job_name, parent_run_id, code_location, nominal_start_time, nominal_end_time, owners, task, run_facets=None)[source]

Emit openlineage event of type START.

Parameters

  • run_id (str) – globally unique identifier of task in dag run

  • job_name (str) – globally unique identifier of task in dag

  • job_description (str) – user provided description of job

  • event_time (str) –

  • parent_job_name (str | None) – the name of the parent job (typically the DAG, but possibly a task group)

  • parent_run_id (str | None) – identifier of job spawning this task

  • code_location (str | None) – file path or URL of DAG file

  • nominal_start_time (str | None) – scheduled time of dag run

  • nominal_end_time (str | None) – following schedule of dag run

  • owners (list[str]) – list of owners of DAG

  • task (airflow.providers.openlineage.extractors.OperatorLineage | None) – metadata container with information extracted from operator

  • run_facets (dict[str, openlineage.client.facet_v2.RunFacet] | None) – custom run facets

complete_task(run_id, job_name, parent_job_name, parent_run_id, end_time, task, run_facets=None)[source]

Emit openlineage event of type COMPLETE.

Parameters

  • run_id (str) – globally unique identifier of task in dag run

  • job_name (str) – globally unique identifier of task between dags

  • parent_job_name (str | None) – the name of the parent job (typically the DAG, but possibly a task group)

  • parent_run_id (str | None) – identifier of job spawning this task

  • end_time (str) – time of task completion

  • task (airflow.providers.openlineage.extractors.OperatorLineage) – metadata container with information extracted from operator

  • run_facets (dict[str, openlineage.client.facet_v2.RunFacet] | None) – additional run facets

fail_task(run_id, job_name, parent_job_name, parent_run_id, end_time, task, error=None, run_facets=None)[source]

Emit openlineage event of type FAIL.

Parameters

  • run_id (str) – globally unique identifier of task in dag run

  • job_name (str) – globally unique identifier of task between dags

  • parent_job_name (str | None) – the name of the parent job (typically the DAG, but possibly a task group)

  • parent_run_id (str | None) – identifier of job spawning this task

  • end_time (str) – time of task completion

  • task (airflow.providers.openlineage.extractors.OperatorLineage) – metadata container with information extracted from operator

  • run_facets (dict[str, openlineage.client.facet_v2.RunFacet] | None) – custom run facets

  • error (str | BaseException | None) – error

  • run_facets – additional run facets

dag_started(dag_id, logical_date, start_date, nominal_start_time, nominal_end_time, owners, run_facets, clear_number, description=None, job_facets=None)[source]
dag_success(dag_id, run_id, end_date, logical_date, clear_number, dag_run_state, task_ids)[source]
dag_failed(dag_id, run_id, end_date, logical_date, clear_number, dag_run_state, task_ids, msg)[source]

Was this entry helpful?