airflow.providers.standard.operators.generic_transfer

Module Contents

Classes

GenericTransfer

Moves data from a connection to another.
class airflow.providers.standard.operators.generic_transfer.GenericTransfer(*, sql, destination_table, source_conn_id, source_hook_params=None, destination_conn_id, destination_hook_params=None, preoperator=None, insert_args=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Moves data from a connection to another.

Assuming that they both provide the required methods in their respective hooks. The source hook needs to expose a get_records method, and the destination a insert_rows method.

This is meant to be used on small-ish datasets that fit in memory.

Parameters

  • sql (str) – SQL query to execute against the source database. (templated)

  • destination_table (str) – target table. (templated)

  • source_conn_id (str) – source connection. (templated)

  • destination_conn_id (str) – destination connection. (templated)

  • preoperator (str | list[str] | None) – sql statement or list of statements to be executed prior to loading the data. (templated)

  • insert_args (dict | None) – extra params for insert_rows method.

template_fields: collections.abc.Sequence[str] = ('source_conn_id', 'destination_conn_id', 'sql', 'destination_table', 'preoperator', 'insert_args')[source]
template_ext: collections.abc.Sequence[str] = ('.sql', '.hql')[source]
template_fields_renderers[source]
ui_color = '#b0f07c'[source]
classmethod get_hook(conn_id, hook_params=None)[source]

Return default hook for this connection id.

Parameters

  • conn_id (str) – connection id

  • hook_params (dict | None) – hook parameters

Returns

default hook for this connection

Return type

airflow.hooks.base.BaseHook

execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?