airflow.providers.common.sql.hooks.lineage

Attributes

log

Classes

SqlJobHookLineageExtra

Keys for the SQL job hook-level lineage extra entry.

Functions

send_sql_hook_lineage(*, context, sql[, ...])

Report a single SQL execution to the hook lineage collector.

Module Contents

airflow.providers.common.sql.hooks.lineage.log[source]
class airflow.providers.common.sql.hooks.lineage.SqlJobHookLineageExtra[source]

Bases: str, enum.Enum

Keys for the SQL job hook-level lineage extra entry.

Reported via get_hook_lineage_collector().add_extra(). KEY is the extra entry key; VALUE__* are the keys inside the value dict (one entry per SQL statement so job_id, SQL text, row count, default_db, etc. stay stitched).

KEY = 'sql_job'[source]
VALUE__SQL_STATEMENT = 'sql'[source]
VALUE__SQL_STATEMENT_PARAMETERS = 'sql_parameters'[source]
VALUE__JOB_ID = 'job_id'[source]
VALUE__ROW_COUNT = 'row_count'[source]
VALUE__DEFAULT_DB = 'default_db'[source]
VALUE__DEFAULT_SCHEMA = 'default_schema'[source]
VALUE__EXTRA = 'extra'[source]
classmethod value_keys()[source]

Value-dict keys only (KEY excluded). Use when iterating or validating the value dict.

airflow.providers.common.sql.hooks.lineage.send_sql_hook_lineage(*, context, sql, sql_parameters=None, cur=None, job_id=None, row_count=None, default_db=None, default_schema=None, extra=None)[source]

Report a single SQL execution to the hook lineage collector.

Call this after running a SQL statement so that hook lineage collectors can associate the execution with the task. Each call produces one extra entry in the collector; when executing multiple statements in one hook run, one should call this function separately for each sql job, so that job_id, SQL text, row count, and other fields stay tied together per statement.

Usable from any hook: pass the hook instance as context. Not limited to DbApiHook subclasses.

Parameters:

  • context (airflow.providers.common.compat.lineage.hook.LineageContext) – Lineage context, typically the hook instance. Must be valid for get_hook_lineage_collector().add_extra(context=..., ...).

  • sql (str | list[str]) – The SQL statement that was executed (or a representative string).

  • sql_parameters (Any) – Optional parameters bound to the statement.

  • cur (Any) – Optional DB-API cursor after execution. If given, job_id is taken from query_id or sfqid when not provided explicitly, and row_count from cur.rowcount when applicable (PEP 249).

  • job_id (str | None) – Explicit job ID; used instead of cursor-derived value when set.

  • row_count (int | None) – Explicit row count; used instead of cursor-derived value when set.

  • default_db (str | None) – Default database/catalog name for this execution context.

  • default_schema (str | None) – Default schema name for this execution context.

  • extra (dict[str, Any] | None) – Optional additional key-value data to attach to this lineage entry.

Was this entry helpful?