airflow.providers.google.cloud.links.dataproc¶

This module contains Google Dataproc links.

Attributes¶

`DATAPROC_BASE_LINK`
`DATAPROC_JOB_LINK`
`DATAPROC_CLUSTER_LINK`
`DATAPROC_WORKFLOW_TEMPLATE_LINK`
`DATAPROC_WORKFLOW_LINK`
`DATAPROC_BATCH_LINK`
`DATAPROC_BATCHES_LINK`
`DATAPROC_JOB_LINK_DEPRECATED`
`DATAPROC_CLUSTER_LINK_DEPRECATED`

Classes¶

`DataprocLink`	Helper class for constructing Dataproc resource link.
`DataprocListLink`	Helper class for constructing list of Dataproc resources link.
`DataprocClusterLink`	Helper class for constructing Dataproc Cluster Link.
`DataprocJobLink`	Helper class for constructing Dataproc Job Link.
`DataprocWorkflowLink`	Helper class for constructing Dataproc Workflow Link.
`DataprocWorkflowTemplateLink`	Helper class for constructing Dataproc Workflow Template Link.
`DataprocBatchLink`	Helper class for constructing Dataproc Batch Link.
`DataprocBatchesListLink`	Helper class for constructing Dataproc Batches List Link.

Functions¶

__getattr__(name)

Module Contents¶

airflow.providers.google.cloud.links.dataproc.__getattr__(name)[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_BASE_LINK = 'https://console.cloud.google.com/dataproc'[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_JOB_LINK = 'https://console.cloud.google.com/dataproc/jobs/{job_id}?region={region}&project={project_id}'[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_CLUSTER_LINK = 'https://console.cloud.google.com/dataproc/clusters/{cluster_id}/monitoring?region={region}&proje...[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_WORKFLOW_TEMPLATE_LINK = 'https://console.cloud.google.com/dataproc/workflows/templates/{region}/{workflow_template_id}?pr...[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_WORKFLOW_LINK = 'https://console.cloud.google.com/dataproc/workflows/instances/{region}/{workflow_id}?project={pr...[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_BATCH_LINK = 'https://console.cloud.google.com/dataproc/batches/{region}/{batch_id}/monitoring?project={project_id}'[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_BATCHES_LINK = 'https://console.cloud.google.com/dataproc/batches?project={project_id}'[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_JOB_LINK_DEPRECATED = 'https://console.cloud.google.com/dataproc/jobs/{resource}?region={region}&project={project_id}'[source]¶

airflow.providers.google.cloud.links.dataproc.DATAPROC_CLUSTER_LINK_DEPRECATED = 'https://console.cloud.google.com/dataproc/clusters/{resource}/monitoring?region={region}&project...[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocLink[source]¶

Bases: airflow.sdk.BaseOperatorLink

Helper class for constructing Dataproc resource link.

Warning

This link is pending to deprecate.

name = 'Dataproc resource'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'conf'[source]¶

static persist(context, task_instance, url, resource)[source]¶

get_link(operator, *, ti_key)[source]¶

Link to external system.

Parameters:

operator (airflow.models.BaseOperator) – The Airflow operator object this link is associated to.
ti_key (airflow.models.taskinstancekey.TaskInstanceKey) – TaskInstance ID to return link for.

Returns:

link to external system

Return type:

str

__attrs_post_init__()[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocListLink[source]¶

Bases: airflow.sdk.BaseOperatorLink

Helper class for constructing list of Dataproc resources link.

Warning

This link is deprecated.

name = 'Dataproc resources'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'list_conf'[source]¶

static persist(context, task_instance, url)[source]¶

get_link(operator, *, ti_key)[source]¶

Link to external system.

Parameters:

operator (airflow.models.BaseOperator) – The Airflow operator object this link is associated to.
ti_key (airflow.models.taskinstancekey.TaskInstanceKey) – TaskInstance ID to return link for.

Returns:

link to external system

Return type:

str

__attrs_post_init__()[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocClusterLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Cluster Link.

name = 'Dataproc Cluster'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'dataproc_cluster'[source]¶

format_str = 'https://console.cloud.google.com/dataproc/clusters/{cluster_id}/monitoring?region={region}&proje...[source]¶

static persist(context, operator, cluster_id, region, project_id)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocJobLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Job Link.

name = 'Dataproc Job'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'dataproc_job'[source]¶

format_str = 'https://console.cloud.google.com/dataproc/jobs/{job_id}?region={region}&project={project_id}'[source]¶

static persist(context, operator, job_id, region, project_id)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocWorkflowLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Workflow Link.

name = 'Dataproc Workflow'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'dataproc_workflow'[source]¶

format_str = 'https://console.cloud.google.com/dataproc/workflows/instances/{region}/{workflow_id}?project={pr...[source]¶

static persist(context, operator, workflow_id, project_id, region)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocWorkflowTemplateLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Workflow Template Link.

name = 'Dataproc Workflow Template'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'dataproc_workflow_template'[source]¶

format_str = 'https://console.cloud.google.com/dataproc/workflows/templates/{region}/{workflow_template_id}?pr...[source]¶

static persist(context, operator, workflow_template_id, project_id, region)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocBatchLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Batch Link.

name = 'Dataproc Batch'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'dataproc_batch'[source]¶

format_str = 'https://console.cloud.google.com/dataproc/batches/{region}/{batch_id}/monitoring?project={project_id}'[source]¶

static persist(context, operator, batch_id, project_id, region)[source]¶

class airflow.providers.google.cloud.links.dataproc.DataprocBatchesListLink[source]¶

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Dataproc Batches List Link.

name = 'Dataproc Batches List'[source]¶: Name of the link. This will be the button name on the task UI.

key = 'dataproc_batches_list'[source]¶

format_str = 'https://console.cloud.google.com/dataproc/batches?project={project_id}'[source]¶

static persist(context, operator, project_id)[source]¶