airflow.providers.google.cloud.operators.dataplex¶

This module contains Google Dataplex operators.

Classes¶

`DataplexCreateTaskOperator`	Creates a task resource within a lake.
`DataplexDeleteTaskOperator`	Delete the task resource.
`DataplexListTasksOperator`	Lists tasks under the given lake.
`DataplexGetTaskOperator`	Get task resource.
`DataplexCreateLakeOperator`	Creates a lake resource within a lake.
`DataplexDeleteLakeOperator`	Delete the lake resource.
`DataplexCreateOrUpdateDataQualityScanOperator`	Creates a DataScan resource.
`DataplexGetDataQualityScanOperator`	Gets a DataScan resource.
`DataplexDeleteDataQualityScanOperator`	Deletes a DataScan resource.
`DataplexRunDataQualityScanOperator`	Runs an on-demand execution of a DataScan.
`DataplexGetDataQualityScanResultOperator`	Gets a Data Scan Job resource.
`DataplexCreateOrUpdateDataProfileScanOperator`	Creates a DataScan Data Profile resource.
`DataplexGetDataProfileScanOperator`	Gets a DataScan DataProfile resource.
`DataplexDeleteDataProfileScanOperator`	Deletes a DataScan DataProfile resource.
`DataplexRunDataProfileScanOperator`	Runs an on-demand execution of a DataScan Data Profile Scan.
`DataplexGetDataProfileScanResultOperator`	Gets a DataScan Data Profile Job resource.
`DataplexCreateZoneOperator`	Creates a Zone resource within a Lake.
`DataplexDeleteZoneOperator`	Deletes a Zone resource. All assets within a zone must be deleted before the zone can be deleted.
`DataplexCreateAssetOperator`	Creates an Asset resource.
`DataplexDeleteAssetOperator`	Deletes an asset resource.
`DataplexCatalogBaseOperator`	Base class for all Dataplex Catalog operators.
`DataplexCatalogCreateEntryGroupOperator`	Create an EntryGroup resource.
`DataplexCatalogGetEntryGroupOperator`	Get an EntryGroup resource.
`DataplexCatalogDeleteEntryGroupOperator`	Delete an EntryGroup resource.
`DataplexCatalogListEntryGroupsOperator`	List EntryGroup resources.
`DataplexCatalogUpdateEntryGroupOperator`	Update an EntryGroup resource.
`DataplexCatalogCreateEntryTypeOperator`	Create an EntryType resource.
`DataplexCatalogGetEntryTypeOperator`	Get an EntryType resource.
`DataplexCatalogDeleteEntryTypeOperator`	Delete an EntryType resource.
`DataplexCatalogListEntryTypesOperator`	List EntryType resources.
`DataplexCatalogUpdateEntryTypeOperator`	Update an EntryType resource.
`DataplexCatalogCreateAspectTypeOperator`	Create an AspectType resource.
`DataplexCatalogGetAspectTypeOperator`	Get an AspectType resource.
`DataplexCatalogListAspectTypesOperator`	List AspectType resources.
`DataplexCatalogUpdateAspectTypeOperator`	Update an AspectType resource.
`DataplexCatalogDeleteAspectTypeOperator`	Delete an AspectType resource.

Module Contents¶

class airflow.providers.google.cloud.operators.dataplex.DataplexCreateTaskOperator(project_id, region, lake_id, body, dataplex_task_id, validate_only=None, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, asynchronous=False, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Creates a task resource within a lake.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the task belongs to.
body (dict[str, Any]) – Required. The Request body contains an instance of Task.
dataplex_task_id (str) – Required. Task identifier.
validate_only (bool | None) – Optional. Only validate the request, but do not perform mutations. The default is false.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
asynchronous (bool) – Flag informing should the Dataplex task be created asynchronously. This is useful for long running creating tasks and waiting on them asynchronously using the DataplexTaskSensor

template_fields = ('project_id', 'dataplex_task_id', 'body', 'validate_only', 'impersonation_chain')[source]¶

template_fields_renderers[source]¶

operator_extra_links[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

body[source]¶

dataplex_task_id[source]¶

validate_only = None[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

asynchronous = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexDeleteTaskOperator(project_id, region, lake_id, dataplex_task_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Delete the task resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the task belongs to.
dataplex_task_id (str) – Required. Task identifier.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ('project_id', 'dataplex_task_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

dataplex_task_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexListTasksOperator(project_id, region, lake_id, page_size=None, page_token=None, filter=None, order_by=None, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Lists tasks under the given lake.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the task belongs to.
page_size (int | None) – Optional. Maximum number of tasks to return. The service may return fewer than this value. If unspecified, at most 10 tasks will be returned. The maximum value is 1000; values above 1000 will be coerced to 1000.
page_token (str | None) – Optional. Page token received from a previous ListZones call. Provide this to retrieve the subsequent page. When paginating, all other parameters provided to ListZones must match the call that provided the page token.
filter (str | None) – Optional. Filter request.
order_by (str | None) – Optional. Order by fields for the result.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ('project_id', 'page_size', 'page_token', 'filter', 'order_by', 'impersonation_chain')[source]¶

operator_extra_links[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

page_size = None[source]¶

page_token = None[source]¶

filter = None[source]¶

order_by = None[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexGetTaskOperator(project_id, region, lake_id, dataplex_task_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Get task resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the task belongs to.
dataplex_task_id (str) – Required. Task identifier.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ('project_id', 'dataplex_task_id', 'impersonation_chain')[source]¶

operator_extra_links[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

dataplex_task_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCreateLakeOperator(project_id, region, lake_id, body, validate_only=None, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, asynchronous=False, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Creates a lake resource within a lake.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
lake_id (str) – Required. Lake identifier.
body (dict[str, Any]) – Required. The Request body contains an instance of Lake.
validate_only (bool | None) – Optional. Only validate the request, but do not perform mutations. The default is false.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
asynchronous (bool) – Flag informing should the Dataplex lake be created asynchronously. This is useful for long-running creating lakes.

template_fields = ('project_id', 'lake_id', 'body', 'validate_only', 'impersonation_chain')[source]¶

template_fields_renderers[source]¶

operator_extra_links[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

body[source]¶

validate_only = None[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

asynchronous = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexDeleteLakeOperator(project_id, region, lake_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Delete the lake resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
lake_id (str) – Required. Lake identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ('project_id', 'lake_id', 'impersonation_chain')[source]¶

operator_extra_links[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCreateOrUpdateDataQualityScanOperator(project_id, region, data_scan_id, body, api_version='v1', retry=DEFAULT, timeout=None, update_mask=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Creates a DataScan resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
body (dict[str, Any] | google.cloud.dataplex_v1.types.DataScan) – Required. The Request body contains an instance of DataScan.
data_scan_id (str) – Required. Data Quality scan identifier.
update_mask (dict | google.protobuf.field_mask_pb2.FieldMask | None) – Mask of fields to update.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

Dataplex data scan id

template_fields = ('project_id', 'data_scan_id', 'body', 'impersonation_chain')[source]¶

template_fields_renderers[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

body[source]¶

update_mask = None[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexGetDataQualityScanOperator(project_id, region, data_scan_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Gets a DataScan resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Quality scan identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

Dataplex data scan

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexDeleteDataQualityScanOperator(project_id, region, data_scan_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Deletes a DataScan resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Quality scan identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

None

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexRunDataQualityScanOperator(project_id, region, data_scan_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, asynchronous=False, fail_on_dq_failure=False, result_timeout=60.0 * 10, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), polling_interval_seconds=10, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Runs an on-demand execution of a DataScan.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Quality scan identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
asynchronous (bool) – Flag informing that the Dataplex job should be run asynchronously. This is useful for submitting long-running jobs and waiting on them asynchronously using the DataplexDataQualityJobStatusSensor
fail_on_dq_failure (bool) – If set to true and not all Data Quality scan rules have been passed, an exception is thrown. If set to false and not all Data Quality scan rules have been passed, execution will finish with success.
result_timeout (float) – Value in seconds for which operator will wait for the Data Quality scan result when the flag asynchronous = False. Throws exception if there is no result found after specified amount of seconds.
polling_interval_seconds (int) – time in seconds between polling for job completion. The value is considered only when running in deferrable mode. Must be greater than 0.
deferrable (bool) – Run operator in the deferrable mode.

Returns:

Dataplex Data Quality scan job id.

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

asynchronous = False[source]¶

fail_on_dq_failure = False[source]¶

result_timeout = 600.0[source]¶

deferrable = True[source]¶

polling_interval_seconds = 10[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]¶

Act as a callback for when the trigger fires - returns immediately.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

class airflow.providers.google.cloud.operators.dataplex.DataplexGetDataQualityScanResultOperator(project_id, region, data_scan_id, job_id=None, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, fail_on_dq_failure=False, wait_for_results=True, result_timeout=60.0 * 10, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), polling_interval_seconds=10, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Gets a Data Scan Job resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Quality scan identifier.
job_id (str | None) – Optional. Data Quality scan job identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
fail_on_dq_failure (bool) – If set to true and not all Data Quality scan rules have been passed, an exception is thrown. If set to false and not all Data Quality scan rules have been passed, execution will finish with success.
wait_for_results (bool) – Flag indicating whether to wait for the result of a job execution or to return the job in its current state.
result_timeout (float) – Value in seconds for which operator will wait for the Data Quality scan result when the flag wait_for_results = True. Throws exception if there is no result found after specified amount of seconds.
polling_interval_seconds (int) – time in seconds between polling for job completion. The value is considered only when running in deferrable mode. Must be greater than 0.
deferrable (bool) – Run operator in the deferrable mode.

Returns:

Dict representing DataScanJob. When the job completes with a successful status, information about the Data Quality result is available.

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain', 'job_id')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

job_id = None[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

fail_on_dq_failure = False[source]¶

wait_for_results = True[source]¶

result_timeout = 600.0[source]¶

deferrable = True[source]¶

polling_interval_seconds = 10[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]¶

Act as a callback for when the trigger fires - returns immediately.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

class airflow.providers.google.cloud.operators.dataplex.DataplexCreateOrUpdateDataProfileScanOperator(project_id, region, data_scan_id, body, api_version='v1', retry=DEFAULT, timeout=None, update_mask=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Creates a DataScan Data Profile resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
body (dict[str, Any] | google.cloud.dataplex_v1.types.DataScan) – Required. The Request body contains an instance of DataScan.
data_scan_id (str) – Required. Data Profile scan identifier.
update_mask (dict | google.protobuf.field_mask_pb2.FieldMask | None) – Mask of fields to update.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

Dataplex data profile id

template_fields = ('project_id', 'data_scan_id', 'body', 'impersonation_chain')[source]¶

template_fields_renderers[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

body[source]¶

update_mask = None[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexGetDataProfileScanOperator(project_id, region, data_scan_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Gets a DataScan DataProfile resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Profile scan identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

Dataplex data profile

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexDeleteDataProfileScanOperator(project_id, region, data_scan_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Deletes a DataScan DataProfile resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Profile scan identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

None

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexRunDataProfileScanOperator(project_id, region, data_scan_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, asynchronous=False, result_timeout=60.0 * 10, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), polling_interval_seconds=10, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Runs an on-demand execution of a DataScan Data Profile Scan.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Profile scan identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
asynchronous (bool) – Flag informing that the Dataplex job should be run asynchronously. This is useful for submitting long-running jobs and waiting on them asynchronously using the DataplexDataProfileJobStatusSensor
result_timeout (float) – Value in seconds for which operator will wait for the Data Profile scan result when the flag asynchronous = False. Throws exception if there is no result found after specified amount of seconds.
polling_interval_seconds (int) – time in seconds between polling for job completion. The value is considered only when running in deferrable mode. Must be greater than 0.
deferrable (bool) – Run operator in the deferrable mode.

Returns:

Dataplex Data Profile scan job id.

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

asynchronous = False[source]¶

result_timeout = 600.0[source]¶

deferrable = True[source]¶

polling_interval_seconds = 10[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]¶

Act as a callback for when the trigger fires - returns immediately.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

class airflow.providers.google.cloud.operators.dataplex.DataplexGetDataProfileScanResultOperator(project_id, region, data_scan_id, job_id=None, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, wait_for_results=True, result_timeout=60.0 * 10, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Gets a DataScan Data Profile Job resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the lake belongs to.
region (str) – Required. The ID of the Google Cloud region that the lake belongs to.
data_scan_id (str) – Required. Data Profile scan identifier.
job_id (str | None) – Optional. Data Profile scan job identifier.
api_version (str) – The version of the api that will be requested for example ‘v1’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
wait_for_results (bool) – Flag indicating whether to wait for the result of a job execution or to return the job in its current state.
result_timeout (float) – Value in seconds for which operator will wait for the Data Profile scan result when the flag wait_for_results = True. Throws exception if there is no result found after specified amount of seconds.

Returns:

Dict representing DataScanJob. When the job completes with a successful status, information about the Data Profile result is available.

template_fields = ('project_id', 'data_scan_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

data_scan_id[source]¶

job_id = None[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

wait_for_results = True[source]¶

result_timeout = 600.0[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]¶

Act as a callback for when the trigger fires - returns immediately.

Relies on trigger to throw an exception, otherwise it assumes execution was successful.

class airflow.providers.google.cloud.operators.dataplex.DataplexCreateZoneOperator(project_id, region, lake_id, body, zone_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Creates a Zone resource within a Lake.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the task belongs to.
body (dict[str, Any] | google.cloud.dataplex_v1.types.Zone) – Required. The Request body contains an instance of Zone.
zone_id (str) – Required. Task identifier.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

Zone

template_fields = ('project_id', 'zone_id', 'body', 'lake_id', 'impersonation_chain')[source]¶

template_fields_renderers[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

body[source]¶

zone_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexDeleteZoneOperator(project_id, region, lake_id, zone_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Deletes a Zone resource. All assets within a zone must be deleted before the zone can be deleted.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the task belongs to.
zone_id (str) – Required. Zone identifier.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

None

template_fields = ('project_id', 'lake_id', 'zone_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

zone_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCreateAssetOperator(project_id, region, lake_id, body, zone_id, asset_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Creates an Asset resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the lake belongs to.
zone_id (str) – Required. Zone identifier.
asset_id (str) – Required. Asset identifier.
body (dict[str, Any] | google.cloud.dataplex_v1.types.Asset) – Required. The Request body contains an instance of Asset.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

Asset

template_fields = ('project_id', 'zone_id', 'asset_id', 'body', 'impersonation_chain')[source]¶

template_fields_renderers[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

body[source]¶

zone_id[source]¶

asset_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexDeleteAssetOperator(project_id, region, lake_id, zone_id, asset_id, api_version='v1', retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Deletes an asset resource.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project that the task belongs to.
region (str) – Required. The ID of the Google Cloud region that the task belongs to.
lake_id (str) – Required. The ID of the Google Cloud lake that the asset belongs to.
zone_id (str) – Required. Zone identifier.
asset_id (str) – Required. Asset identifier.
api_version (str) – The version of the api that will be requested for example ‘v3’.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Additional metadata that is provided to the method.
gcp_conn_id (str) – The connection ID to use when fetching connection info.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Returns:

None

template_fields = ('project_id', 'zone_id', 'asset_id', 'impersonation_chain')[source]¶

project_id[source]¶

region[source]¶

lake_id[source]¶

zone_id[source]¶

asset_id[source]¶

api_version = 'v1'[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogBaseOperator(project_id, location, gcp_conn_id='google_cloud_default', retry=DEFAULT, timeout=None, metadata=(), impersonation_chain=None, *args, **kwargs)[source]¶

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Base class for all Dataplex Catalog operators.

Parameters:

project_id (str) – Required. The ID of the Google Cloud project where the service is used.
location (str) – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id (str) – Optional. The connection ID to use to connect to Google Cloud.
retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout (float | None) – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata (collections.abc.Sequence[tuple[str, str]]) – Optional. Additional metadata that is provided to the method.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str] = ('project_id', 'location', 'gcp_conn_id', 'impersonation_chain')[source]¶

project_id[source]¶

location[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

retry[source]¶

timeout = None[source]¶

metadata = ()[source]¶

property hook: airflow.providers.google.cloud.hooks.dataplex.DataplexHook[source]¶

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogCreateEntryGroupOperator(entry_group_id, entry_group_configuration, validate_request=False, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Create an EntryGroup resource.

See also

For more information on how to use this operator, take a look at the guide: Create an EntryGroup

Parameters:

entry_group_id (str) – Required. EntryGroup identifier.
entry_group_configuration (google.cloud.dataplex_v1.types.EntryGroup | dict) – Required. EntryGroup configuration. For more details please see API documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.entryGroups#EntryGroup
validate_request (bool) – Optional. If set, performs request validation, but does not actually execute the request.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

entry_group_id[source]¶

entry_group_configuration[source]¶

validate_request = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogGetEntryGroupOperator(entry_group_id, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Get an EntryGroup resource.

See also

For more information on how to use this operator, take a look at the guide: Get an EntryGroup

Parameters:

entry_group_id (str) – Required. EntryGroup identifier.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

entry_group_id[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogDeleteEntryGroupOperator(entry_group_id, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Delete an EntryGroup resource.

See also

For more information on how to use this operator, take a look at the guide: Delete an EntryGroup

Parameters:

entry_group_id (str) – Required. EntryGroup identifier.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

entry_group_id[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogListEntryGroupsOperator(page_size=None, page_token=None, filter_by=None, order_by=None, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

List EntryGroup resources.

See also

For more information on how to use this operator, take a look at the guide: List EntryGroups

Parameters:

filter_by (str | None) – Optional. Filter to apply on the list results.
order_by (str | None) – Optional. Fields to order the results by.
page_size (int | None) – Optional. Maximum number of EntryGroups to return on the page.
page_token (str | None) – Optional. Token to retrieve the next page of results.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str] = ('project_id', 'location', 'gcp_conn_id', 'impersonation_chain')[source]¶

operator_extra_links[source]¶

page_size = None[source]¶

page_token = None[source]¶

filter_by = None[source]¶

order_by = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogUpdateEntryGroupOperator(entry_group_id, entry_group_configuration, update_mask=None, validate_request=False, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Update an EntryGroup resource.

See also

For more information on how to use this operator, take a look at the guide: Update an EntryGroup

Parameters:

project_id – Required. The ID of the Google Cloud project that the task belongs to.
location – Required. The ID of the Google Cloud region that the task belongs to.
update_mask (list[str] | google.protobuf.field_mask_pb2.FieldMask | None) – Optional. Names of fields whose values to overwrite on an entry group. If this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied.
entry_group_id (str) – Required. ID of the EntryGroup to update.
entry_group_configuration (dict | google.cloud.dataplex_v1.types.EntryGroup) – Required. The updated configuration body of the EntryGroup. For more details please see API documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.entryGroups#EntryGroup
validate_only – Optional. The service validates the request without performing any mutations.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
gcp_conn_id – Optional. The connection ID to use when fetching connection info.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

entry_group_id[source]¶

entry_group_configuration[source]¶

update_mask = None[source]¶

validate_request = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogCreateEntryTypeOperator(entry_type_id, entry_type_configuration, validate_request=False, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Create an EntryType resource.

See also

For more information on how to use this operator, take a look at the guide: Create an EntryType

Parameters:

entry_type_id (str) – Required. EntryType identifier.
entry_type_configuration (google.cloud.dataplex_v1.types.EntryType | dict) – Required. EntryType configuration. For more details please see API documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.entryGroups#EntryGroup
validate_request (bool) – Optional. If set, performs request validation, but does not actually execute the request.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

entry_type_id[source]¶

entry_type_configuration[source]¶

validate_request = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogGetEntryTypeOperator(entry_type_id, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Get an EntryType resource.

See also

For more information on how to use this operator, take a look at the guide: Get an EntryType

Parameters:

entry_type_id (str) – Required. EntryType identifier.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

entry_type_id[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogDeleteEntryTypeOperator(entry_type_id, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Delete an EntryType resource.

See also

For more information on how to use this operator, take a look at the guide: Delete an EntryType

Parameters:

entry_type_id (str) – Required. EntryType identifier.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

entry_type_id[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogListEntryTypesOperator(page_size=None, page_token=None, filter_by=None, order_by=None, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

List EntryType resources.

See also

For more information on how to use this operator, take a look at the guide: List EntryTypes

Parameters:

filter_by (str | None) – Optional. Filter to apply on the list results.
order_by (str | None) – Optional. Fields to order the results by.
page_size (int | None) – Optional. Maximum number of EntryTypes to return on the page.
page_token (str | None) – Optional. Token to retrieve the next page of results.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str] = ('project_id', 'location', 'gcp_conn_id', 'impersonation_chain')[source]¶

operator_extra_links[source]¶

page_size = None[source]¶

page_token = None[source]¶

filter_by = None[source]¶

order_by = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogUpdateEntryTypeOperator(entry_type_id, entry_type_configuration, update_mask=None, validate_request=False, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Update an EntryType resource.

See also

For more information on how to use this operator, take a look at the guide: Update an EntryType

Parameters:

project_id – Required. The ID of the Google Cloud project that the task belongs to.
location – Required. The ID of the Google Cloud region that the task belongs to.
update_mask (list[str] | google.protobuf.field_mask_pb2.FieldMask | None) – Optional. Names of fields whose values to overwrite on an entry group. If this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied.
entry_type_id (str) – Required. ID of the EntryType to update.
entry_type_configuration (dict | google.cloud.dataplex_v1.types.EntryType) – Required. The updated configuration body of the EntryType. For more details please see API documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.entryGroups#EntryGroup
validate_only – Optional. The service validates the request without performing any mutations.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
gcp_conn_id – Optional. The connection ID to use when fetching connection info.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

entry_type_id[source]¶

entry_type_configuration[source]¶

update_mask = None[source]¶

validate_request = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogCreateAspectTypeOperator(aspect_type_id, aspect_type_configuration, validate_request=False, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Create an AspectType resource.

See also

For more information on how to use this operator, take a look at the guide: Create an AspectType

Parameters:

aspect_type_id (str) – Required. AspectType identifier.
aspect_type_configuration (google.cloud.dataplex_v1.types.AspectType | dict) – Required. AspectType configuration. For more details please see API documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.aspectTypes#AspectType
validate_request (bool) – Optional. If set, performs request validation, but does not actually execute the request.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

aspect_type_id[source]¶

aspect_type_configuration[source]¶

validate_request = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogGetAspectTypeOperator(aspect_type_id, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Get an AspectType resource.

See also

For more information on how to use this operator, take a look at the guide: Get an AspectType

Parameters:

aspect_type_id (str) – Required. AspectType identifier.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

aspect_type_id[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogListAspectTypesOperator(page_size=None, page_token=None, filter_by=None, order_by=None, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

List AspectType resources.

See also

For more information on how to use this operator, take a look at the guide: List AspectTypes

Parameters:

filter_by (str | None) – Optional. Filter to apply on the list results.
order_by (str | None) – Optional. Fields to order the results by.
page_size (int | None) – Optional. Maximum number of AspectTypes to return on the page.
page_token (str | None) – Optional. Token to retrieve the next page of results.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str] = ('project_id', 'location', 'gcp_conn_id', 'impersonation_chain')[source]¶

operator_extra_links[source]¶

page_size = None[source]¶

page_token = None[source]¶

filter_by = None[source]¶

order_by = None[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogUpdateAspectTypeOperator(aspect_type_id, aspect_type_configuration, update_mask=None, validate_request=False, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Update an AspectType resource.

See also

For more information on how to use this operator, take a look at the guide: Update an AspectType

Parameters:

project_id – Required. The ID of the Google Cloud project that the task belongs to.
location – Required. The ID of the Google Cloud region that the task belongs to.
update_mask (list[str] | google.protobuf.field_mask_pb2.FieldMask | None) – Optional. Names of fields whose values to overwrite on an entry group. If this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied.
aspect_type_id (str) – Required. ID of the AspectType to update.
aspect_type_configuration (dict | google.cloud.dataplex_v1.types.AspectType) – Required. The updated configuration body of the AspectType. For more details please see API documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.aspectTypes#AspectType
validate_only – Optional. The service validates the request without performing any mutations.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
gcp_conn_id – Optional. The connection ID to use when fetching connection info.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

operator_extra_links[source]¶

aspect_type_id[source]¶

aspect_type_configuration[source]¶

update_mask = None[source]¶

validate_request = False[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.dataplex.DataplexCatalogDeleteAspectTypeOperator(aspect_type_id, *args, **kwargs)[source]¶

Bases: DataplexCatalogBaseOperator

Delete an AspectType resource.

See also

For more information on how to use this operator, take a look at the guide: Delete an AspectType

Parameters:

aspect_type_id (str) – Required. AspectType identifier.
project_id – Required. The ID of the Google Cloud project where the service is used.
location – Required. The ID of the Google Cloud region where the service is used.
gcp_conn_id – Optional. The connection ID to use to connect to Google Cloud.
retry – Optional. A retry object used to retry requests. If None is specified, requests will not be retried.
timeout – Optional. The amount of time, in seconds, to wait for the request to complete. Note that if retry is specified, the timeout applies to each individual attempt.
metadata – Optional. Additional metadata that is provided to the method.
impersonation_chain – Optional. Service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields: collections.abc.Sequence[str][source]¶

aspect_type_id[source]¶

execute(context)[source]¶

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.