• Home
• airflow.providers.alibaba
• airflow.providers.alibaba.cloud
• airflow.providers.alibaba.cloud.hooks
• airflow.providers.alibaba.cloud.hooks.maxcompute

airflow.providers.alibaba.cloud.hooks.maxcompute

Attributes

RT

Classes

MaxComputeHook

Interact with Alibaba MaxCompute (previously known as ODPS).

Functions

fallback_to_default_project_endpoint(func)

Provide fallback for MaxCompute project and endpoint to be used as a decorator.

Module Contents

airflow.providers.alibaba.cloud.hooks.maxcompute.RT

airflow.providers.alibaba.cloud.hooks.maxcompute.fallback_to_default_project_endpoint(func)

Provide fallback for MaxCompute project and endpoint to be used as a decorator.

If the project or endpoint is None it will be replaced with the project from the connection extra definition.

Parameters:

func (Callable[Ellipsis, RT]) – function to wrap

Returns:

result of the function call

Return type:

Callable[Ellipsis, RT]

class airflow.providers.alibaba.cloud.hooks.maxcompute.MaxComputeHook(maxcompute_conn_id='maxcompute_default', **kwargs)

Bases: airflow.providers.alibaba.cloud.hooks.base_alibaba.AlibabaBaseHook

Interact with Alibaba MaxCompute (previously known as ODPS).

Parameters:

maxcompute_conn_id (str) – The connection ID to use when fetching connection info.

conn_name_attr = 'maxcompute_conn_id'

default_conn_name = 'maxcompute_default'

conn_type = 'maxcompute'

hook_name = 'MaxCompute'

maxcompute_conn_id = 'maxcompute_default'

classmethod get_connection_form_widgets()

Return connection widgets to add to connection form.

classmethod get_ui_field_behaviour()

Return custom field behaviour.

property project: str

Returns project ID.

Returns:

ID of the project

Return type:

str

property endpoint: str

Returns MaxCompute Endpoint.

Returns:

Endpoint of the MaxCompute project

Return type:

str

get_client(*, project, endpoint)

Get an authenticated MaxCompute ODPS Client.

Parameters:

• project_id – Project ID for the project which the client acts on behalf of.

• location – Default location for jobs / datasets / tables.

run_sql(*, sql, project=None, endpoint=None, priority=None, running_cluster=None, hints=None, aliases=None, default_schema=None, quota_name=None)

Run a given SQL statement in MaxCompute.

The method will submit your SQL statement to MaxCompute and return the corresponding task Instance object.

See also

https://pyodps.readthedocs.io/en/latest/base-sql.html#execute-sql

Parameters:

• sql (str) – The SQL statement to run.

• project (str | None) – The project ID to use.

• endpoint (str | None) – The endpoint to use.

• priority (int | None) – The priority of the SQL statement ranges from 0 to 9, applicable to projects with the job priority feature enabled. Takes precedence over the odps.instance.priority setting from hints. Defaults to 9. See https://www.alibabacloud.com/help/en/maxcompute/user-guide/job-priority for details.

• running_cluster (str | None) – The cluster to run the SQL statement on.

• hints (dict[str, Any] | None) – Hints for setting runtime parameters. See https://pyodps.readthedocs.io/en/latest/base-sql.html#id4 and https://www.alibabacloud.com/help/en/maxcompute/user-guide/flag-parameters for details.

• aliases (dict[str, str] | None) – Aliases for the SQL statement.

• default_schema (str | None) – The default schema to use.

• quota_name (str | None) – The quota name to use. Defaults to project default quota if not specified.

Returns:

The MaxCompute task instance.

Return type:

odps.models.Instance

get_instance(*, instance_id, project=None, endpoint=None)

Get a MaxCompute task instance.

See also

https://pyodps.readthedocs.io/en/latest/base-instances.html#instances

Parameters:

• instance_id (str) – The ID of the instance to get.

• project (str | None) – The project ID to use.

• endpoint (str | None) – The endpoint to use.

Returns:

The MaxCompute task instance.

Raises:

ValueError – If the instance does not exist.

Return type:

odps.models.Instance

stop_instance(*, instance_id, project=None, endpoint=None)

Stop a MaxCompute task instance.

Parameters:

• instance_id (str) – The ID of the instance to stop.

• project (str | None) – The project ID to use.

• endpoint (str | None) – The endpoint to use.

Previous Next

Was this entry helpful?