airflow.providers.google.cloud.operators.cloud_composer

Module Contents

Classes

CloudComposerEnvironmentLink

Helper class for constructing Cloud Composer Environment Link.

CloudComposerEnvironmentsLink

Helper class for constructing Cloud Composer Environment Link.

CloudComposerCreateEnvironmentOperator

Create a new environment.

CloudComposerDeleteEnvironmentOperator

Delete an environment.

CloudComposerGetEnvironmentOperator

Get an existing environment.

CloudComposerListEnvironmentsOperator

List environments.

CloudComposerUpdateEnvironmentOperator

Update an environment.

CloudComposerListImageVersionsOperator

List ImageVersions for provided location.

CloudComposerRunAirflowCLICommandOperator

Run Airflow command for provided Composer environment.

Attributes

CLOUD_COMPOSER_BASE_LINK

CLOUD_COMPOSER_DETAILS_LINK

CLOUD_COMPOSER_ENVIRONMENTS_LINK

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Cloud Composer Environment Link.

name = 'Cloud Composer Environment'[source]
key = 'composer_conf'[source]
format_str[source]
static persist(operator_instance, context)[source]

Bases: airflow.providers.google.cloud.links.base.BaseGoogleLink

Helper class for constructing Cloud Composer Environment Link.

name = 'Cloud Composer Environment List'[source]
key = 'composer_conf'[source]
format_str[source]
static persist(operator_instance, context)[source]
class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerCreateEnvironmentOperator(*, project_id, region, environment_id, environment, gcp_conn_id='google_cloud_default', impersonation_chain=None, retry=DEFAULT, timeout=None, metadata=(), deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), pooling_period_seconds=30, **kwargs)[source]

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Create a new environment.

Parameters

  • project_id (str) – Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) – Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) – Required. The ID of the Google Cloud environment that the service belongs to.

  • environment (google.cloud.orchestration.airflow.service_v1.types.Environment | dict) – The environment to create.

  • gcp_conn_id (str) –

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Designation of what errors, if any, should be retried.

  • timeout (float | None) – The timeout for this request.

  • metadata (collections.abc.Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

  • deferrable (bool) – Run operator in the deferrable mode

  • pooling_period_seconds (int) – Optional: Control the rate of the poll for the result of deferrable run. By default, the trigger will poll every 30 seconds.

template_fields = ('project_id', 'region', 'environment_id', 'environment', 'impersonation_chain')[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event)[source]
class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerDeleteEnvironmentOperator(*, project_id, region, environment_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), pooling_period_seconds=30, **kwargs)[source]

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Delete an environment.

Parameters

  • project_id (str) – Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) – Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) – Required. The ID of the Google Cloud environment that the service belongs to.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Designation of what errors, if any, should be retried.

  • timeout (float | None) – The timeout for this request.

  • metadata (collections.abc.Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) –

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • deferrable (bool) – Run operator in the deferrable mode

  • pooling_period_seconds (int) – Optional: Control the rate of the poll for the result of deferrable run. By default, the trigger will poll every 30 seconds.

template_fields = ('project_id', 'region', 'environment_id', 'impersonation_chain')[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event)[source]
class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerGetEnvironmentOperator(*, project_id, region, environment_id, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Get an existing environment.

Parameters

  • project_id (str) – Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) – Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) – Required. The ID of the Google Cloud environment that the service belongs to.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Designation of what errors, if any, should be retried.

  • timeout (float | None) – The timeout for this request.

  • metadata (collections.abc.Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) –

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ('project_id', 'region', 'environment_id', 'impersonation_chain')[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerListEnvironmentsOperator(*, project_id, region, page_size=None, page_token=None, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

List environments.

Parameters

  • project_id (str) – Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) – Required. The ID of the Google Cloud region that the service belongs to.

  • page_size (int | None) – The maximum number of environments to return.

  • page_token (str | None) – The next_page_token value returned from a previous List request, if any.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Designation of what errors, if any, should be retried.

  • timeout (float | None) – The timeout for this request.

  • metadata (collections.abc.Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) –

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ('project_id', 'region', 'impersonation_chain')[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerUpdateEnvironmentOperator(*, project_id, region, environment_id, environment, update_mask, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), pooling_period_seconds=30, **kwargs)[source]

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Update an environment.

Parameters

  • project_id (str) – Required. The ID of the Google Cloud project that the service belongs to.

  • region (str) – Required. The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) – Required. The ID of the Google Cloud environment that the service belongs to.

  • environment (dict | google.cloud.orchestration.airflow.service_v1.types.Environment) – A patch environment. Fields specified by the updateMask will be copied from the patch environment into the environment under update.

  • update_mask (dict | google.protobuf.field_mask_pb2.FieldMask) – Required. A comma-separated list of paths, relative to Environment, of fields to update. If a dict is provided, it must be of the same form as the protobuf message FieldMask

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Designation of what errors, if any, should be retried.

  • timeout (float | None) – The timeout for this request.

  • metadata (collections.abc.Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) –

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • deferrable (bool) – Run operator in the deferrable mode

  • pooling_period_seconds (int) – Optional: Control the rate of the poll for the result of deferrable run. By default, the trigger will poll every 30 seconds.

template_fields = ('project_id', 'region', 'environment_id', 'impersonation_chain')[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event)[source]
class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerListImageVersionsOperator(*, project_id, region, page_size=None, page_token=None, include_past_releases=False, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, **kwargs)[source]

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

List ImageVersions for provided location.

Parameters

  • request – The request object. List ImageVersions in a project and location.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Designation of what errors, if any, should be retried.

  • timeout (float | None) – The timeout for this request.

  • metadata (collections.abc.Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) –

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

template_fields = ('project_id', 'region', 'impersonation_chain')[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.google.cloud.operators.cloud_composer.CloudComposerRunAirflowCLICommandOperator(*, project_id, region, environment_id, command, retry=DEFAULT, timeout=None, metadata=(), gcp_conn_id='google_cloud_default', impersonation_chain=None, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), poll_interval=10, **kwargs)[source]

Bases: airflow.providers.google.cloud.operators.cloud_base.GoogleCloudBaseOperator

Run Airflow command for provided Composer environment.

Parameters

  • project_id (str) – The ID of the Google Cloud project that the service belongs to.

  • region (str) – The ID of the Google Cloud region that the service belongs to.

  • environment_id (str) – The ID of the Google Cloud environment that the service belongs to.

  • command (str) – Airflow command.

  • retry (google.api_core.retry.Retry | google.api_core.gapic_v1.method._MethodDefault) – Designation of what errors, if any, should be retried.

  • timeout (float | None) – The timeout for this request.

  • metadata (collections.abc.Sequence[tuple[str, str]]) – Strings which should be sent along with the request as metadata.

  • gcp_conn_id (str) – The connection ID used to connect to Google Cloud Platform.

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

  • deferrable (bool) – Run operator in the deferrable mode

  • poll_interval (int) – Optional: Control the rate of the poll for the result of deferrable run. By default, the trigger will poll every 10 seconds.

template_fields = ('project_id', 'region', 'environment_id', 'command', 'impersonation_chain')[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event)[source]

Was this entry helpful?