airflow.providers.alibaba.cloud.operators.maxcompute¶

This module contains Alibaba Cloud MaxCompute operators.

Classes¶

MaxComputeSQLOperator

Executes an SQL statement in MaxCompute.

Module Contents¶

class airflow.providers.alibaba.cloud.operators.maxcompute.MaxComputeSQLOperator(*, sql, project=None, endpoint=None, priority=None, running_cluster=None, hints=None, aliases=None, default_schema=None, quota_name=None, alibabacloud_conn_id='alibabacloud_default', cancel_on_kill=True, **kwargs)[source]¶

Bases: airflow.models.BaseOperator

Executes an SQL statement in MaxCompute.

Waits for the SQL task instance to complete and returns instance id.

Parameters:

sql (str) – The SQL statement to run.
project (str | None) – The project ID to use.
endpoint (str | None) – The endpoint to use.
priority (int | None) – The priority of the SQL statement ranges from 0 to 9, applicable to projects with the job priority feature enabled. Takes precedence over the odps.instance.priority setting from hints. Defaults to 9. See https://www.alibabacloud.com/help/en/maxcompute/user-guide/job-priority for details.
running_cluster (str | None) – The cluster to run the SQL statement on.
hints (dict[str, str] | None) – Hints for setting runtime parameters. See https://pyodps.readthedocs.io/en/latest/base-sql.html#id4 and https://www.alibabacloud.com/help/en/maxcompute/user-guide/flag-parameters for details.
aliases (dict[str, str] | None) – Aliases for the SQL statement.
default_schema (str | None) – The default schema to use.
quota_name (str | None) – The quota name to use. Defaults to project default quota if not specified.
alibabacloud_conn_id (str) – The connection ID to use. Defaults to alibabacloud_default if not specified.
cancel_on_kill (bool) – Flag which indicates whether to stop running instance or not when task is killed. Default is True.

template_fields: collections.abc.Sequence[str] = ('sql', 'project', 'endpoint', 'priority', 'running_cluster', 'hints', 'aliases',...[source]¶

template_ext: collections.abc.Sequence[str] = ('.sql',)[source]¶

template_fields_renderers[source]¶

operator_extra_links[source]¶

sql[source]¶

project = None[source]¶

endpoint = None[source]¶

priority = None[source]¶

running_cluster = None[source]¶

hints = None[source]¶

aliases = None[source]¶

default_schema = None[source]¶

quota_name = None[source]¶

alibabacloud_conn_id = 'alibabacloud_default'[source]¶

cancel_on_kill = True[source]¶

hook: airflow.providers.alibaba.cloud.hooks.maxcompute.MaxComputeHook | None = None[source]¶

instance: odps.models.Instance | None = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

on_kill()[source]¶

Override this method to clean up subprocesses when a task instance gets killed.

Any use of the threading, subprocess or multiprocessing module within an operator needs to be cleaned up, or it will leave ghost processes behind.